Experimental Evaluation of the Variation in Effectiveness for DC, FPC and MC/DC Test Criteria

Given a test criterion, the number of test-sets satisfying the criterion may be very large, with varying fault detection effectiveness. This paper presents an experimental evaluation of the variation in fault detection effectiveness of all the test-sets for a given control-flow test criterion and a Boolean specification. The exhaustive experimental approach complements the earlier empirical studies that adopted analysis of some test-sets using random selection techniques. Three industrially used control-flow testing criteria, Decision Coverage (DC), Full Predicate Coverage (FPC) and Modified Condition/Decision Coverage (MC/DC) have been analysed against four types of faults. The Boolean specifications used were taken from a past research paper and also generated randomly. To ensure that it is the test-set property that influences the effectiveness and not the test-set size, a variation of DC, Decision Coverage/Random (DC/R), has also been considered against FPC and MC/DC criteria. In addition, a further analysis of variation in average effectiveness with respect to number of conditions in the decision has been done. The empirical results show that the MC/DC criterion is more reliable and stable in comparison to DC, DC/R and FPC.